The present invention relates generally to real-time event processing, and more particularly to methods, apparatus and systems for processing real-time events in applications such as telecommunications and computer networks.
Many event-processing systems have real-time performance requirements that cannot be met by conventional general-purpose database management systems. In telecommunications applications, for example, a variety of adjunct switching services such as debit-based billing, number mapping, call forwarding, and local-number portability involve event processing during the critical call-connection phase of a telephone call. To meet the real-time requirements of the network, the service time for such events generally must not exceed a few milliseconds. However, with conventional database technology, the costs of invoking a structured query language (SQL) operation over a client-server interface, or the costs associated with a single access to secondary storage, can account for hundreds of milliseconds. As a consequence, performance goals on the order of a few milliseconds may already be unattainable even before the costs of the event processing logic are taken into account.
These limitations have led to the use of custom database systems for many high-performance real-time event processing applications. Custom systems are typically tightly coupled to their particular applications, and are tuned to the specific requirements of those applications. These solutions generally work well in practice, and can provide close to optimal performance. However, they also have a number of serious disadvantages. For example, the cost of developing and maintaining custom systems can be high, and generally cannot be amortized across a number of different applications. Moreover, custom systems are frequently inflexible. It can become difficult or even iimpossible to adapt a custom system to unforeseen or evolving requirements.
A need therefore exists for an improved real-time event processing system which provides the performance benefits of custom database systems, without sacrificing the flexibility and maintainability typically associated with conventional general-purpose database systems.
The invention provides a general-purpose real-time event processing system (EPS) which avoids the problems associated with custom systems. The EPS in an illustrative embodiment meets real-time performance goals through the use of one or more real-time analysis engines (RAEs) operating in conjunction with a main-memory storage manager as its underlying database system. The main-memory storage manager offers transactional access to persistent data, but at the speed of a main-memory system. The RAEs store subscription information in accordance with a subscription model that serves as a basis for flexible, low-overhead, real-time event processing.
In accordance with the invention, the subscription information indicating the service or services that should be executed for a given event is stored in the form of a subscription table in the memory portion of the main-memory database system. The subscription table may be in the form of a compact subscription table which includes a subscriber identifier, a start timestamp, an end timestamp, and a subscription term specified using a run-time interpretable subscription language. At any particular point in time, the compact subscription table includes at most one relevant entry for each subscriber. The compact subscription table may be generated by applying translation and update propagation operations to a more complex subscription table generated by the system application. In operation, a RAE determines if a given event is a non-subscribed event or a subscribed event, directly invokes a corresponding event handler if the event is a non-subscribed event, and accesses the stored subscription information to determine at least one event handler if the event is a subscribed event. If the event is a subscribed event, the RAE may use the stored subscription information to determine a set of event handlers and an order in which to call them to process the event.
A real-time EPS in accordance with the invention provides a critical path for event processing that is specifically designed for high performance, while also retaining many desirable features of conventional database systems, including high-level, declarative programming interfaces, and the well-known transactional correctness properties of atomicity, consistency, isolation and durability (ACID). These features of the invention enhance the reliability, robustness, usability and maintainability of the real-time EPS and the applications built thereon.
The invention can be used in conjunction with any event processing application, including, for example, telecommunications, electronic commerce, and Internet service provisioning applications. For example, in a telecommunications application, the invention can be configured to provide a basis for features such as enhanced billing systems, fraud detection and prevention, local-number portability, settlements among service providers and real-time traffic analysis.